Several file formats are available when either creating a new file or converting an existing one. Each of these formats have pros and cons.
About real-time recording: certain formats require a lot of computing resources when encoding. On most of the recent phones (ARMv7), TapeMachine can record directly into all formats below. On other phones, certain formats, such as OGG, are only available when converting, not when creating a new file.
You may record by appending to an existing file or by overwriting parts of the sound, but only with uncompressed formats such as WAV and AIFF.
Such formats as OGG and FLAC are open, powerful, and recommended. However, not everyone has the codecs which are necessary to read files in these formats. On computers, one can use the VLC application for this purpose.
Also note that, although direct OGG editing is supported, it is highly recommended to perform edit operations using an uncompressed format, and convert afterwards if needed.